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The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )^ Responsive to communication(s) filed on 10 April 2000 . 
2a)Q This action is FINAL. 2b)IEI This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle t 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) ^ Claim(s) 1^7 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 1^7 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) ^ The specification is objected to by the Examiner. 

10) E3 The drawing(s) filed on 10 April 2000 is/are: a)D accepted or b)E3 objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

11) D The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)ISI All b)D Some*c)D None of: 

1 Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attach ment(s) 

1 ) ^ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) Paper No(s). . 

2) D Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-1 52) 

3) Information Disclosure Statement(s) (PTO-1 449) Paper No(s) 7 . 6) □ Other: 
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DETAILED ACTION 



Priority 



1 . Receipt is acknowledged of papers submitted under 35 U.S. C. 1 1 9(a)-(d), which papers 
have been placed of record in the file. 



2. The listing of references in the specification is not a proper information disclosure 
statement (page 8, line 20-21). 37 CFR 1.98(b) requires a list of all patents, publications, or 
other information submitted for consideration by the Office, and MPEP § 609 A(l) states, "the 
list may not be incorporated into the specification but must be submitted in a separate paper." 
Therefore, unless the references have been cited by the examiner on form PTO-892, they have 
not been considered. 



3. The drawings are objected to under 37 CFR 1.83(a). The drawings must show every 
feature of the invention specified in the claims. Therefore, the "direction of the optical axis of 
said image pickup means with respect to the reference object" must be shown or the feature(s) 
canceled from the claim(s). No new matter should be entered. 

A proposed drawing correction or corrected drawings are required in reply to the Office 
action to avoid abandonment of the application. The objection to the drawings will not be held 
in abeyance. 

4. The drawings are objected to under 37 CFR 1.83(a) because they fail to show a 
coordinate system with respect to the image pickup means and the workpiece as described in the 
specification. Any structural detail that is essential for a proper understanding of the disclosed 



Information Disclosure Statement 



Drawings 
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invention should be shown in the drawing. MPEP § 608.02(d). A proposed drawing correction 
or corrected drawings are required in reply to the Office action to avoid abandonment of the 
application. The objection to the drawings will not be held in abeyance. 
5. Figures 5, 6, and 11-13 should be designated by a legend such as --Prior Art- because 
only that which is old is illustrated. See MPEP § 608.02(g). A proposed drawing correction or 
corrected, drawings are required in reply to the Office action to avoid abandonment of the 
application. The objection to the drawings will not be held in abeyance. 



6. The disclosure is objected to because of the following informalities: page 8, line 2-3 
refers to "image memory" with reference numeral 26. This contradicts the labeling with numeral 
36 in Figure 6. 

Appropriate correction is required. 



7. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

8. Claims 1-3 are rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. 

9. The claims are generally narrative and indefinite, failing to conform with current U.S. 
practice. They appear to be a literal translation into English from a foreign document and are 
replete with grammatical and idiomatic errors. Lines 1-2 of claims 1 and 2 state, "an object or 
article having the same shape as that of the object." This is confusing and it is unclear what the 



Specification 



Claim Rejections - 35 USC § 112 
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applicant intended this to mean. In addition, line 4 of claims 1 and 2 describe a process of 
matching a plurality of teaching models with the reference object to determine the three- 
dimensional position and/or posture of the object, but the rest of the claims seem to describe the 
process of registering a teaching model using a plurality of image pickup positions. It is unclear 
where one process starts and the other begins. Claim 3 states that "said teaching model is part of 
the image data of the reference object." It is unclear what the applicant intended this to mean. It 
would make more sense to say, for instance, "said image data of the reference object is part of 
said teaching model," however, this is already described in the last two lines of claim 2 and 
would not constitute a further limiting dependent claim. 

Claim Rejections - 35 USC § 102 

10. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

11. Claims 1, 3, and 4 are rejected under 35 U.S.C. 102(b) as being anticipated by Suzuki 
(USPN 4,835,450). 

12. Regarding claim 1, Suzuki discloses a teaching model generating device for image 
processing, in which an object or an article having the same shape as that of the object is selected 
as a reference object (column 3, line 66 to column 4, line 9), and three-dimensional position 
and/or posture of said object is recognized by carrying out matching processing of a plurality of 
teaching models (column 4, line 25-39), which were generated and stored in advance on the basis 
of the respective image data produced by taking the image of said reference object from a 
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plurality of directions, with the image data including the object, wherein one of the reference 
object and the image pickup means is fixed in place, while the other is fixed to a movable part of 
a robot or is grasped with a hand of the robot (either moving the pair of CCD cameras 11a and 
1 lb around fixed sample 10, or rotating sample 10 in front of the fixed cameras), and said robot 
is operated for positioning to a plurality of image pickup positions where direction of the optical 
axis of said image pickup means with respect to the reference object is different from one 
another respectively (column 3, line 66 to column 4, line 9); so that the image data respectively 
obtained at each of said image pickup positions is stored as a teaching model (column 4, line 32- 



13. Regarding claim 3, Suzuki discloses that said teaching model is a part of the image data 
of the reference object (column 4, line 32-39). 

14. Regarding claim 4, Suzuki discloses that said teaching model is composed of data 
obtained by performing an image processing (3-D position detecting circuit 16) on the image 
data of the reference object (column 4, line 5-9). 



15. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 



16. Claims 2, 5, and 6 are rejected under 35 U.S.C. 103(a) as being unpatentable over Suzuki 
in view of Werth et al. (USPN 4,504,970), hereafter Werth. 



39). 



Claim Rejections - 35 USC § 103 
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17. Regarding claim 2, Suzuki discloses a teaching model generating device for image 
processing, in which an object or an article having the same shape as that of the object is selected 
as a reference object, and three- dimensional position and/or posture of said object is recognized 
by carrying out matching processing of a plurality of teaching models, which were generated and 
stored in advance on the basis of the respective image data produced by taking the image of said 
reference object from a plurality of directions, with the image data including the object, wherein 
a robot is operated for positioning to a plurality of image pickup positions where direction of the 
optical axis of said image pickup means with respect to the reference object is different from one 
another respectively, so that the image data respectively obtained at each of said image pickup 
positions is stored as a teaching model (see discussion of claim 1 above). 

18. Suzuki does not, however, disclose that the reference object is fixed to a movable part of 
a first robot or is grasped with a hand of the first robot, and an image pickup means is fixed to a 
movable part a second robot or is grasped with a hand of the second robot. Werth discloses a 
training controller for pattern processing system wherein it is suggested that an application could 
utilize two robot arms, one which holds a camera which visually guides it to observe a precise 
assembly point and a second which brings a tool or assembly within the visual field of the 
camera where it is visually guided through an operation (column 5, line 12-17). It would have 
been obvious to one of ordinary skill in the art at the time the invention was made to utilize two 
robot arms, one to hold a camera and one to hold a tool or assembly as taught by Werth in order 
to visually guide the movement of the camera and the workpiece around one another and to 
provide more degrees of freedom to provide more views from different directions of the 
workpiece. 
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19. Regarding claim 5, Suzuki discloses that said teaching model is generated for every 
direction in which said image pickup means took the image of said reference object (column 4, 
line 2-9) and said teaching model is stored in association with the information on the direction 
(column 4, line 32-39). 

20. Regarding claim 6, Suzuki discloses that said image pickup means is a camera (column 3, 
line 30-34). 

21. Claim 7 is rejected under 35 U.S.C. 103(a) as being unpatentable over Suzuki in view of 
Werth as applied to claim 2 above, and further in view of Ninomiya et al. (USPN 4,61 1,292), 
hereafter Ninomiya. 

22. Suzuki does not disclose that said image pickup means is a three-dimensional visual 
sensor whose image pickup means measures the distance between the image pickup means and a 
plurality of points on the object. Ninomiya discloses a robot vision system including a three- 
dimensional visual sensor whose image pickup means measures the distance between the image 
pickup means and a plurality of points on the object (column 4, line 28-49). It would have been 
obvious to one of ordinary skill in the art at the time the invention was made to utilize a three- 
dimensional visual sensor as taught by Ninomiya in order to determine the position and posture 
of an object without operation or accuracy being effected by contrast, color, or surface condition 
of the object (column 10, line 35-39). 

Conclusion 

23. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

o USPN 4,73 1 ,853 to Hata et al. discloses a three-dimensional vision system. 
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• USPN 6,101,455 to Davis discloses an automatic calibration of cameras and 
structured light sources. 

• USPN 4,785,528 to Soderberg discloses a robotic work positioning system. 

• USPN 6,1 14,824 to Watanabe discloses a calibration method for a visual sensor. 

• USPN 5,148,591 to Pryor discloses a vision target based assembly. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ryan J Hesseltine whose telephone number is 703-306-4069. 
The examiner can normally be reached on Monday - Friday, 8 AM - 4:30 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on 703-308-6604. The fax phone numbers for the 
organization where this application or proceeding is assigned are 703-872-9314 for regular 
communications and 703-872-9314 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is 703-306-0377. 



rjh 

December 23, 2002 
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